Eye tracking system and method

ABSTRACT

An eye tracking system includes an imaging device, a memory, and a controller. The imaging device is configured to be mounted on a wearable device. The wearable device includes at least a first lens that is light transmissive and is positioned in front of at least a first eye of a user that is wearing the wearable device. The imaging device has a field of view that captures an inner surface of the first lens and a reflection of the first eye on the inner surface. The memory is configured to store program instructions. The controller is operably connected to the memory and the imaging device. The program instructions are executable by the controller to analyze image data generated by the imaging device and to detect a position of a pupil of the first eye in the reflection based on the analysis of the image data.

FIELD

The present disclosure relates to eye tracking technology.

BACKGROUND OF THE INVENTION

Eye tracking technology is used in various applications, such as virtualreality gaming, augmented (or mixed) reality devices, and even inautomobiles to monitor driver awareness. Eye tracking technologyattempts to discern where the subject is looking (or gazing) based onthe positioning of the subject's eye or eyes. A current method of eyetracking involves emitting infrared (IR) light towards the subject'seye(s), such as into the pupil. One drawback of this method is that anIR light source may be relatively expensive and/or complex. A devicethat uses IR light for eye tracking may require a significant number ofcomponents. Another, potentially more significant drawback is thatshining the IR light at the eye(s) may be relatively invasive,uncomfortable, and/or distracting for the subject. Furthermore, thelight source may be in the field of view of the subject. The lightsource may undesirably obstruct the subject's view of objects in thesurrounding environment, and/or may distract the subject by drawing theeye away from the surrounding environment.

A need remains for a system and method that detect the position of asubject's pupil for reliable, accurate eye tracking, without using IRlight and without positioning hardware in front of the subject's eyes.

SUMMARY

In accordance with an embodiment, an eye tracking system is providedthat includes an imaging device, a memory, and a controller. The imagingdevice is configured to be mounted on a wearable device. The wearabledevice includes at least a first lens that is light transmissive and ispositioned in front of at least a first eye of a user that is wearingthe wearable device. The imaging device has a field of view thatcaptures an inner surface of the first lens and a reflection of thefirst eye on the inner surface. The memory is configured to storeprogram instructions. The controller is operably connected to the memoryand the imaging device. The program instructions are executable by thecontroller to analyze image data generated by the imaging device and todetect a position of a pupil of the first eye in the reflection based onthe analysis of the image data.

Optionally, the imaging device is configured to generate the image datain a visible wavelength range. Optionally, the imaging device is mountedsuch that the field of view does not directly capture the first eye ofthe user. The eye tracking system may include a light source configuredto be mounted to the wearable device and to emit light towards the innersurface of the first lens for providing the reflection. The controllermay be configured to detect the position of the pupil in the reflectionrelative to one or more of the first eye in the reflection, the firstlens in the reflection, or the field of view of the imaging device.Optionally, the eye tracking system includes the wearable device, whichis eyeglasses or goggles.

Optionally, the controller is configured to segment the image data to aforeground environment and an external environment. The foregroundenvironment includes the inner surface of the first lens and thereflection. The external environment is disposed beyond the first lens.The controller may be configured to determine a location in the externalenvironment to which a gaze of the first eye is directed based on theposition of the pupil and a correlation between the foregroundenvironment and the external environment. The controller may beconfigured to analyze the image data and identify an object at thelocation in the external environment to which the gaze is directed. Thecontroller may be configured to perform an operation based on the objectthat is identified. For example, the controller may be configured tocontrol a display device to display graphic indicia on a display screenvisible to the user. The graphic indicia relates to the object that isidentified. The eye tracking system may include the display device whichis configured to be mounted on the wearable device and to display thegraphic indicia relating to the object on at least the first lens forviewing by the user.

Optionally, the controller is configured to detect the position of thepupil of the first eye in the reflection by calculating coordinates of acenter point of the pupil in the reflection within a foregroundreference frame in the image data. The controller may be configured todetermine a location in an external environment to which a gaze of thefirst eye is directed by inputting the coordinates of the center pointof the pupil into a transfer function that represents a correlationbetween the foreground reference frame and an external reference frame.

In accordance with an embodiment, a method is provided that includesanalyzing image data generated by an imaging device mounted on awearable device. The wearable device includes at least a first lens thatis light transmissive and positioned in front of at least a first eye ofa user that is wearing the wearable device. The imaging device has afield of view that captures an inner surface of the first lens and areflection of the first eye on the inner surface. The method includesdetecting a position of a pupil of the first eye in the reflection basedon the analysis of the image data.

Optionally, the method includes removably attaching the imaging deviceto a frame of the wearable device. Optionally, detecting the position ofthe pupil includes detecting the position of the pupil in the reflectionrelative to one or more of the first eye in the reflection, the firstlens in the reflection, or the field of view of the imaging device.

Optionally, the method includes segmenting the image data between aforeground environment and an external environment. The foregroundenvironment includes the inner surface of the first lens and thereflection. The external environment is disposed beyond the first lensrelative to the imaging device. The method includes determining alocation in the external environment to which a gaze of the first eye isdirected based on the position of the pupil and a correlation betweenthe foreground environment and the external environment. Optionally, themethod includes identifying an object at the location in the externalenvironment to which the gaze is directed. The method may furtherinclude controlling a display device to display graphic indicia on adisplay screen visible to the user. The graphic indicia relates to theobject that is identified.

In accordance with an embodiment, a computer program product is providedthat includes a non-transitory computer readable storage medium. Thenon-transitory computer readable storage medium includes computerexecutable code configured to be executed by one or more processors toanalyze image data generated by an imaging device mounted on a wearabledevice. The wearable device includes at least a first lens that is lighttransmissive and positioned in front of at least a first eye of a userthat is wearing the wearable device. The imaging device has a field ofview that captures an inner surface of the first lens and a reflectionof the first eye on the inner surface. The computer executable code isconfigured to be executed by one or more processors to detect a positionof a pupil of the first eye in the reflection based on the analysis ofthe image data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a wearable device that includes an eye trackingsystem according to an embodiment.

FIG. 2 is a block diagram of the eye tracking system according to anembodiment.

FIG. 3A is a diagram showing the eye tracking device, a lens of awearable device, and an eye of a user that is wearing the wearabledevice according to an embodiment.

FIG. 3B illustrates a reflection of the user's eye on an inner surfaceof the lens according to the eye position shown in FIG. 3A.

FIG. 4A is a diagram showing the eye tracking device, the lens of thewearable device, and the eye of the user gazing in a different directionrelative to the gaze direction in FIG. 3A.

FIG. 4B illustrates the reflection of the user's eye on the innersurface of the lens according to the eye position shown in FIG. 4A.

FIG. 5 illustrates an image generated by the imaging device of the eyetracking system according to an embodiment.

FIG. 6 illustrates the same image as FIG. 5 , and identifies a secondlocation that the user is gazing relative to the image.

FIG. 7 illustrates a lens of the wearable device as viewed by a userduring operation of the eye tracking system according to an embodiment.

FIG. 8 . is a flow chart of a method of eye tracking according to anembodiment.

DETAILED DESCRIPTION

It will be readily understood that the components of the embodiments asgenerally described and illustrated in the figures herein, may bearranged and designed in a wide variety of different configurations inaddition to the described example embodiments. Thus, the following moredetailed description of the example embodiments, as represented in thefigures, is not intended to limit the scope of the embodiments, asclaimed, but is merely representative of example embodiments.

Reference throughout this specification to “one embodiment” or “anembodiment” (or the like) means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment. Thus, appearances of the phrases “in oneembodiment” or “in an embodiment” or the like in various placesthroughout this specification are not necessarily all referring to thesame embodiment.

Furthermore, the described features, structures, or characteristics maybe combined in any suitable manner in one or more embodiments. In thefollowing description, numerous specific details are provided to give athorough understanding of embodiments. One skilled in the relevant artwill recognize, however, that the various embodiments can be practicedwithout one or more of the specific details, or with other methods,components, materials, etc. In other instances, well-known structures,materials, or operations are not shown or described in detail to avoidobfuscation. The following description is intended only by way ofexample, and simply illustrates certain example embodiments.

References herein to “machine learning” and “artificial intelligence”refer to algorithms that learn from various automatic or manualfeedback, such as observations and/or data. The artificial intelligencealgorithms may be adjusted over multiple iterations based on theobservations and/or data. For example, the artificial intelligencealgorithms may be adjusted by supervised learning, unsupervisedlearning, and/or reinforcement learning (e.g., customer feedback).Non-limiting examples of artificial intelligence algorithms includedecision trees, K-means, deep learning, artificial neural networks,and/or the like.

References herein to “subject” and “user” refer to a person whose eye(e.g., pupil) is being tracked by the system and method disclosedherein. For example, the user may be a person that is wearing a wearabledevice on which the eye tracking system, or a portion thereof, ismounted.

Embodiments described herein disclose an eye tracking system and methodthat detects the position of a user's pupil for reliable, accurate eyetracking. The system and method disclosed herein may not emit orotherwise utilize IR light. The system and method may not locatehardware in front of the subject's eyes, such that the hardware does notimpede the user's vision. The eye tracking system and method disclosedherein detects a position of the pupil based on a reflection of theuser's eye on an inner surface of a lens. The lens may be a lighttransmissive lens through which the user see through to visualize theexternal environment surrounding the user. In one example application,the lens is a lens of eyeglasses worn by the user. The eye trackingsystem includes an imaging device that is positioned to capture theinner surface of the lens in image data generated by the imaging device.The imaging device may be mounted on a wearable device that is worn bythe user such that the imaging device is out of the way of the user'sdirect line of sight to the external environment. For example, theimaging device may be mounted to the side of the user's eye, such asproximate to the user's ear, rather than in front of the eye. In theeyeglasses example, the imaging device may be mounted to one temple ofthe eyeglasses frame.

The eye tracking system and method analyze the image data generated bythe imaging device and detect the position of the pupil of the user'seye based on the reflection of the pupil on the lens inner surface, asdepicted in the image data. The system and method may monitor theposition of the pupil over time to detect eye movements of the user. Inan embodiment, the eye tracking system and method may determine a gazedirection of the user into the external environment based on theposition of the pupil and the image data generated by the imagingdevice. For example, the image data may depict a portion of the externalenvironment that is beyond the lens. The depicted portion of theexternal environment may be through the light transmissive lens and/oroutside of a perimeter of the lens.

The eye tracking system and method may use a correlation between thedetected position of the pupil in the reflection and the depictedportion of the external environment to determine a location in theexternal environment to which the user's gaze is directed. In anembodiment, the eye tracking system and method may analyze the imagedata and identify an object present at the location in the externalenvironment indicated by the gaze direction. For example, the object maybe identified as a sign, a natural formation, a plant, an animal, abuilding, a vehicle, a phone, and/or the like. The eye tracking systemand method may perform an operation based on the identity of the objectto which the user is looking (e.g., gazing). For example, the system andmethod may present information about the object to the user. Theinformation may be displayed on the lens itself or on a discrete displaydevice. In another example, the system and method may alert the user viadisplaying a flashing light or message, emitting a warning tone, or thelike, if the identified object indicates that the user's gaze is outsideof a designated viewing zone. This example may be relevant for ensuringthat vehicle operators (e.g., drivers) are alert and focused the routeand the operation of the vehicle. Additional details and exampleapplications of the eye tracking system and method are described hereinwith reference to the appended figures.

FIG. 1 illustrates a wearable device 100 that includes an eye trackingsystem 102 according to an embodiment. The wearable device 100 is wornby a user on the user's head. The wearable device 100 includes a frame104 and at least one lens 106 held by the frame 104. The wearable device100 in the illustrated embodiment includes two lenses 106, each locatedin front of a different corresponding eye of the user when the device100 is properly worn. For example, the wearable device 100 may beeyeglasses. The lenses 106 may be light transmissive, such that the usercan view the external environment through the lenses 106. The lenses 106may be transparent, or at least translucent, like typical eyeglasslenses.

The external environment as used herein refers to portions of arespective field of view beyond the lenses 106 and the user's face. Auser wearing eyeglasses may be able to view the lenses, the rim of theglasses surrounding the lenses, a portion of the temples connected tothe rims, a tip of the user's nose, and/or the like. These elements arereferred to herein as within a foreground environment. The user wouldalso be able to see objects in the external environment, beyond theglasses and the user's face, such as the ground, the sky, naturalwildlife, other people, buildings, vehicles, computers, the user's armsand legs, and/or the like.

In an embodiment, the wearable device 100 may be augmented (or mixed)reality “smart” glasses that provide a head-up display. The eye trackingsystem 102 may display information to the user on one or both lenses 106to supplement the visual information inherently provided by the externalenvironment. Alternatively, the wearable device 100 does not present ahead-up display on either of the lenses 106 of the eyeglasses. In otherembodiments, the wearable device 100 may be goggles with a lens thatlaterally extends across both eyes of the user. One example type ofgoggles is a virtual reality (VR) headset. The wearable device may beother types of wearable devices in other embodiments, such as a monocle,a rear view mirror assembly for a bicycle helmet, a light transmissiveface shield, and/or the like.

The eye tracking system 102 includes an imaging device 108. The imagingdevice 108 is mounted to the frame 104 of the wearable device 100. Inthe illustrated embodiment, the imaging device 108 is mounted to atemple 110 of the eyeglasses, which extends from the lens 106 and rim112 back to the user's ear. The imaging device 108 may be located to theside of the user's eyes. The imaging device 108 may be outside of theuser's direct line of sight (e.g., only visible in the user's peripheralvision or not event visible in the user's peripheral vision). Theimaging device 108 may be oriented such that a field of view 118 of theimaging device 108 substantially overlap the field of view of at leastone of the user's eyes. For example, the imaging device 108 may beoriented in a forward direction, similar to the general viewingdirection of the user's eyes. As described herein, the imaging device108 captures the user's pupil in a reflection on an inner surface of thelens 106 that is closest to the imaging device 108.

In an embodiment, additional components of the eye tracking system 102may be packaged with the imaging device 108 in or on a housing 114 thatis mounted to the frame 104 as shown in FIG. 1 . For example, acontroller, a memory device, and a power source may be disposed withinthe housing 114. The components of the eye tracking system 102 mountedto the wearable device 100 define an eye tracking device 116 orassembly. In an embodiment, all of the components of the eye trackingsystem 102 described with reference to FIG. 2 may be integrated withinthe eye tracking device 116. The components may be relatively small andtightly packed to provide a compact form factor. The mounting locationand compact size of the eye tracking system 102 may allow the user'svision of the external environment to be unimpeded by the eye trackingsystem 102. In an alternative embodiment, one or more of the componentsof the eye tracking system 102 may not be mounted on the wearable device100, and may communicate with components on the wearable device 100 viaa wired or wireless communication link.

Optionally, the eye tracking device 116 may be designed to be removableto enable installation on different wearable devices. For example, auser may want to retrofit an existing pair of eyeglasses with the eyetracking device 116, and another user may want to switch the eyetracking device 116 between different wearable devices for different useapplications. The eye tracking device 116 may include a clip, fasteners,or the like, to enable secure, but non-permanent, coupling of the eyetracking device 116 to the wearable device 100.

The eye tracking system 102 in FIG. 1 has a single eye tracking device116 with a single imaging device 108. The eye tracking system 102detects the position of the pupil of a first eye of the user based on areflection along a first lens 106A of the two lenses 106. The eyetracking system 102 may not detect the position of the pupil of theuser's other eye in the illustrated embodiment. In an alternativeembodiment, the eye tracking system 102 may include two eye trackingdevices (e.g., two imaging devices) for monitoring the position of bothpupils. In another embodiment, the imaging device 108 may be positionedand oriented to capture the reflection of both eyes in the image datagenerated by the imaging device 108, which enables dual eye tracking.For example, the imaging device 108 may be installed on the wearabledevice between the user's eyes such that the field of view encompassesreflected views of both eyes. The imaging device 108 between the user'seyes may be disposed above or below the eyes to avoid interfering withthe user's vision, or at least limit the interference.

FIG. 2 is a block diagram of the eye tracking system 102 according to anembodiment. The eye tracking system 102 includes a controller 202 thatperforms some or all of the operations described herein to detect theposition of the pupil of a user's eye based on a reflection in the lens106. The eye tracking system 102 may also include the imaging device108, a light source 204, an input device 206, a display device 208, apower source 210, and a communication device 212. The controller 202 isoperably connected to the other components of the eye tracking system102 via wired and/or wireless communication links to permit thetransmission of information and/or commands in the form of signals. Forexample, the controller 202 may generate control signals that aretransmitted to the other components to control operation of thecomponents. The eye tracking system 102 may have additional componentsthat are not shown in FIG. 2 . In an alternative embodiment, the eyetracking system 102 may lack one or more of the components that areshown in FIG. 2 , such as the communication device 212, the input device206, or the display device 208 as examples.

The controller 202 represents hardware circuitry that includes and/or isconnected with one or more processors 214 (e.g., one or moremicroprocessors, integrated circuits, microcontrollers, fieldprogrammable gate arrays, etc.). The controller 202 includes and/or isconnected with a tangible and non-transitory computer-readable storagemedium (e.g., data storage device), referred to herein as memory device216 or simply as memory. The memory 216 may store program instructions(e.g., software) that are executed by the one or more processors 214 toperform the operations described herein. The program instructions mayinclude one or more algorithms utilized by the one or more processors214 to analyze the image data generated by the imaging device 108 anddetect the position (and movement) of the pupil in the reflected view ofthe user's eye on the lens 106. The one or more algorithms stored in thememory 216 may include image segmentation and processing algorithms foridentifying objects depicted in the image data. The program instructionsmay dictate actions to be performed by the one or more processors 214,such as generating control signals to display information related toidentified objects on the display device 208. The memory 216 may storeinformation that is used by the processors 214, such as a database forstoring calibration data. The calibration data may be used to correlatethe position of the pupil as detected in the reflection on the lens 106with a location in the external environment to which the user is gazing(e.g., looking). The calibration data may include a transfer function, alook-up table, and/or the like. The memory 216 optionally may storeapplications, such as various application program interfaces (APIs) thatlink to cloud hosting services, via the communication device 212, foraccessing information from remote storage devices (e.g., servers).

The imaging device 108 is an optical sensor that generates optical dataof the environment within the field of view 118 (shown in FIG. 1 ) ofthe imaging device 108. The optical data is referred to herein as imagedata. The image data is conveyed to the controller 202 for imageanalysis. The image data may be stored, at least temporarily, in amemory device 216 of the controller 202. The imaging device 108 may be acamera, such as a video camera that generates image data at a specificframe rate (e.g., number of individual images generated per second). Theimaging device 108 may generate the image data in the visible wavelengthrange of the electromagnetic spectrum. In an embodiment, the imagingdevice 108 does not generate image data based on light in the IRwavelength range of the spectrum.

The light source 204 emits light towards the inner surface of the lens106 to provide or enhance the reflection of the user's eye on the innersurface, relative to not actively emitting light onto the inner surface.The light source 204 may be integrated with or coupled to the imagingdevice 108 of the eye tracking device 116 that is mounted to thewearable device 100, as shown in FIG. 1 . In an embodiment, the lightsource 204 does not emit IR light. The light source 204 emits light inone or more other wavelength ranges of the electromagnetic spectrum,such as the visible wavelength range.

The input device 206 receives user input selections for interacting withthe eye tracking system 102. The input device 206 may include orrepresent one or more physical buttons, a microphone, a touch sensitivepad, a switch, or the like. A user may actuate the input device 206 toselectively activate and deactivate the eye tracking device 116 on thewearable device 100. In an alternative embodiment, the eye trackingsystem 102 may lack the input device 206. User input selections, such asto activate and deactivate the device 116, may be communicated to thecontroller 202 wirelessly via the communication device 212, rather thangenerated using the input device 206.

The display device 208 presents graphic indicia, such as text and/orsymbols, on a display screen for viewing by the user. In an embodiment,the display device 208 presents the graphic indicia on at least one lens106 of the wearable device 100, such as to provide a head-up display.Alternatively, the display device 208 may present the graphic indicia onanother display screen that is separate from the lenses 106, such as thedisplay screen of a smartphone, a smartwatch, a tablet computer, orother computer device operably connected to the controller 202.

The power source 210 supplies electrical energy to power the operationsof the eye tracking system 102. The power source 210 may include one ormore batteries, capacitors, or other energy storage devices. The powersource 210 may include rechargeable batteries for extended operationallife of the power source 210 before replacement, while maintaining acompact form factor of the eye tracking device 116.

The communication device 212 represents hardware circuitry that cancommunicate electrical signals via wireless communication pathwaysand/or wired conductive pathways. The communication device 212 mayinclude transceiving circuitry, one or more antennas, and the like, forwireless communication. The communication device 106 may communicatewith a cellular tower, a modem, a router, and/or the like.

FIG. 3A is a diagram 300 illustrating the eye tracking device 116, alens 106 of a wearable device, and an eye 302 of a user that is wearingthe wearable device. The wearable device itself is not depicted in FIG.3A. The lens 106 is located in front of the user's eye 302, such thatthe user visualizes the external environment 306 by gazing through thelens 106. The eye tracking device 116 is mounted to the wearable deviceto the side of the eye 302, rather than in front of the eye 302. The eyetracking device 116 may be positioned outside of the user's direct fieldof view. For example, the user may only be able to indirectly see theeye tracking device 116 via a reflection on an inner surface 307 of thelens 106. The eye tracking device 116 includes at least the imagingdevice 108 and the light source 204 shown in FIG. 2 . The field of view118 of the imaging device 108 captures at least a portion of the innersurface 307 of the lens 106. In an embodiment, the field of view 118does not directly capture the user's eye 302. The imaging device 108indirectly captures the eye 302 within a reflection on the inner surface307 of the lens 106. The light source 204 may emit light towards theinner surface 307 for providing the reflection (or enhancing the naturalreflection). For example, the arrow 310 in FIG. 3A represents a beam orray of light emitted from the light source 204 that reflects off theinner surface 307 and impinges upon a pupil 304 of the user's eye 302.The arrow 312 in FIG. 3A represents a gaze direction of the user's eye302, indicating where the user is looking. The gaze direction extends toobjects in the external environment 306.

FIG. 3B illustrates a reflection 320 of the user's eye 302 on the innersurface 307 of the lens 106 according to the eye position shown in FIG.3A. For example, the pupil 304 in FIG. 3A is directed to the left ofcenter, and the pupil 304 in the reflection 320 in FIG. 3B is also leftof center relative to the perimeter of the eye 302. The reflected viewin FIG. 3B may be a portion of the image data generated by the imagingdevice 108 of the eye tracking device 116. For example, the imagingdevice 108 generates image data that includes the reflection 320, amongother subject matter captured by the imaging device 108.

In an embodiment, the controller 202 of the eye tracking system 102analyzes the image data generated by the imaging device 108, and detectsa position of the pupil 304 of the eye 302 in the reflection 320 basedon the analyzed image data. The controller 202 may monitor the positionof the pupil 304 in the reflection 320 over time to track eye movements(e.g., changes in the gaze direction). For example, the imaging device108 may periodically or repeatedly generate image data over time, andthe controller 202 may detect an updated position of the pupil 304 inthe reflection 320 in response to receiving the new, updated image data.The controller 202 may compare the current (e.g., most recent) detectedposition of the pupil 304 to one or more preceding positions of thepupil 304 to track movement of the eye 302.

In an embodiment, the controller 202 detects the position of the pupil304 relative to a reference object or frame. For example, the positionof the pupil 304 may be represented as coordinates in a reference frameand/or coordinates defined relative to a reference point. The referenceobject or frame may include the eye 302, the lens 106, and/or the fieldof view 118 of the imaging device 108. For example, the perimeter of theeye 302 and/or the perimeter of the lens 106 within the reflection 320may be used to define a frame of reference. FIG. 3B shows a referenceframe 308 that is defined based on the perimeter of the eye 302 in thereflection 320. For example, the controller 202 may perform imageanalysis to identify perimeter edges of the eye 302, and may generate abounding box to enclose the eye 302. The size and location of thebounding box (e.g., the reference frame 308) may be based on edgedetection of another image analysis technique. Once the reference frame308 is established, the controller 202 may divide the area of thereference frame 308 into positional coordinates to define a coordinatesystem.

The controller 202 may perform image analysis to identify the pupil 304within the reflection 320. For example, the controller 202 may identifypixels that likely represent the pupil based on a wavelength (e.g.,color) of the pixels, edge detection by comparing dark colored pixelsnext to lighter colored pixels, a circular shape of a collection of thepixels, and/or the like. Optionally, other techniques may be used toidentify the pupil 304 in the image data, such as using machine visionand trained object detection. The controller 202 may include or access aneural network that is trained via labeled training images to detect theeye 302 in the reflection 320, the pupil 304 in the reflection 320,and/or other features of interest present in the image data. The neuralnetwork may be trained to detect specific shapes, colors, and/or colorcontrasts that would indicate the presence of the eye 302, the pupil304, or the like.

Upon identifying the pupil 304 in the image data, the controller 202 maycalculate positional coordinates of the pupil 304 relative to thereference frame 308. The positional coordinates may be two-dimensional(e.g., x, y) coordinates within the positional coordinate system definedby the reference frame 308. The controller 202 may reduce the pupil 304to a single point position, and then calculate the positionalcoordinates at that single point position. The controller 202 maydetermine a center point 322 (e.g., centroid) of the pupil 304 based onthe image data. For example, the controller 202 may generate a boundingbox that surrounds and encloses the pupil 304 based on the detectededges of the pupil 304, and then may calculate the center point 322 asthe center of the area of the bounding box. The positional coordinatesof the center point 322 are referred to herein as (x₁, y₁). Optionally,the coordinate system may be defined based on a vertical “y” axis 326and a lateral “x” axis 328 which intersect at an origin point 324 at thecenter of the reference frame 308. The coordinates of the center point322 of the pupil 304 may be relative to the origin point 324. Forexample, the x₁ value may be negative because the center point 322 is tothe left of the origin 324, and the y₁ value may also be negativebecause the center point 322 is below the origin 324. The coordinates ofthe center point 322 represent the position of the pupil 304 as detectedby the controller 202.

FIG. 4A is a diagram 400 illustrating the eye tracking device 116, thelens 106 of the wearable device, and the eye 302 of the user, with theeye 302 gazing in a different direction relative to the gaze direction312 in FIG. 3A. In FIG. 4A, the eye 302 is looking along a gazedirection 402 towards the right. The eye tracking system 102 (e.g., thedevice 116) detects the new position of the pupil 304 in the same way asdescribed with reference to FIGS. 3A and 3B. FIG. 4B illustrates thereflection 320 of the user's eye 302 on the inner surface 307 of thelens 106 according to the eye position shown in FIG. 4A. For example,the pupil 304 in FIG. 4A is directed to the right of center, and thepupil 304 in the reflection 320 in FIG. 4B is also right of centerrelative to the eye 302.

The controller 202 may perform image analysis to identify the pupil 304within the reflection 320. Then the controller 202 detects the positionof the pupil 304 relative to the reference frame 308. The controller 202may calculate positional coordinates of the pupil 304 relative to thereference frame 308. The coordinates of the center point 322 of the eye302 at the second position shown in FIG. 4B are (x₂, y₂). The x₂ valuemay be positive because the center point 322 is to the right of theorigin 324, and the y₂ value may also be positive because the centerpoint 322 is above the origin 324. The coordinates (x₂, y₂) of thecenter point 322 represent the position of the pupil 304 in the updatedposition of the eye as detected by the controller 202.

The controller 202 may calculate the magnitude of eye movement betweenthe first position shown in FIGS. 3A and 3B and the second positionshown in FIGS. 4A and 4B. For example, the controller 202 may calculatethe distance of a line segment between the first coordinates (x₁, y₁)and the second coordinates (x₂, y₂) to determine the magnitude.Optionally, the controller 202 may determine a vector from thepositional coordinates (x₁, y₁) to the positional coordinates (x₂, y₂),which represents the distance and direction of pupil movement.

In an embodiment, the eye tracking system 102 may perform one or moreoperations based on the detected position of the pupil 304 in thereflection 320. For example, the controller 202 may calculate the gazedirection (e.g., directions 312, 402) of the eye and/or a location inthe external environment 306 to which the user's gaze is directed, basedon the position of the pupil 304.

The image data generated by the imaging device 118 depicts the externalenvironment 306 beyond the lens 106, as shown in FIGS. 3A and 4A. In anembodiment, the controller 202 may segment the image data between aforeground environment 404 and the external environment 306. Theforeground environment 404 may include portions of the wearable device,including the frame and the reflection 320 on the lens 106, that aredepicted in the image data. The foreground environment 404 may alsoinclude portions of the user's face (e.g., tip of nose, cheek, etc.)captured in the image data. The external environment 404 includesdepicted objects that are physically located beyond the lens 106relative to the imaging device 118. For example, some objects of theexternal environment 306 may be depicted through the light transmissivelens 106.

The controller 202 may segment the image data to differentiate imagedata that represents the foreground environment 404 from image data therepresents the external environment 306. The controller 202 may performthe segmentation based on one or more segmentation algorithms stored inthe memory 216. The segmentation operation may involve edge detection,boundary analysis, and/or the like. The segmentation operation may bebased on trained learning techniques to identify objects that are withinthe foreground environment 404. The controller 202 may use machinelearning (e.g., artificial intelligence), such as a neural network, thatis trained to identify image data depicting the frame of the wearabledevice, the tip of the user's nose, the reflection 320, and/or the like,and may classify that image data as being within the foregroundenvironment 404. The reference frame 308 of the reflection 320 may bereferred to as a foreground reference frame. The origin point 324 in thereference frame 308 may be referred to as a foreground reference point.Remaining image data in the field of view may be classified by thecontroller 202 as being within the external environment 306.

In an embodiment, the controller 202 may determine or utilize acorrelation between the foreground environment 404 and the externalenvironment 306. The controller 202 may use the correlation with thedetected position of the pupil 304 to determine a location in theexternal environment to which the gaze of the user is directed. Thecorrelation may represent a transfer function between the foregroundreference frame 308 and a reference frame in the external environment306. Optionally, the field of view 118 of the imaging device 108 maydefine the boundaries for an external reference frame. For example, theexternal reference frame may represent the two-dimensional planecaptured within the field of view 118 at a distance from the imagingdevice 108 that represents the focal length of the imaging device 108.

FIG. 5 illustrates an image 500 generated by the imaging device 108 ofthe eye tracking system 102 according to an embodiment. The image 500shows both the foreground environment 404, including a portion of thewearable device 100, and the external environment 306. The perimeter ofthe image 500 may be defined by the field of view 118 of the imagingdevice 108, shown in FIGS. 3A and 4A. The controller 202 may define anexternal reference frame 502 based on the perimeter of the image 500(e.g., the field of view 118). The controller 202 may divide the area ofthe image frame into positional coordinates to define a secondcoordinate system (which is different from the first coordinate systemwithin the foreground reference frame 308). For example, the secondcoordinate system may include a vertical “y” axis 512 and a lateral “x”axis 514 that intersect at an origin point 516. A transfer function maybe generated to correlate coordinates of the foreground reference frame308 (e.g., the reflected view of the eye 302) to coordinates of theexternal reference frame 502. The transfer function correlates theforeground environment 404 to the external environment 306 depicted inthe image data.

In one embodiment, the transfer function may be generated with userassistance during a guided set-up procedure to calibrate the eyetracking system 102 for a given user. For example, during the set-upprocedure, the user may wear the wearable device 100 with the eyetracking device 116 mounted thereon, and may be presented with a stockimage to view on the display device 208. The stock image may represent ascene with different objects. The controller 202 may prompt the user todirect the user's gaze to a first object in the scene. The user may usethe input device 206 to indicate that the user is looking at the firstobject. In response to receiving the input signal, the controller 202may detect the position of the pupil 304 of the user's eye 302 whenviewing the first object. Then, the controller 202 may prompt the userto gaze at a second object in the scene, and may detect the new positionof the user's pupil when looking at the second object. This process mayrepeat for different objects in the scene. The stock image may have aknown coordinates in the external reference frame 502 for each of theobjects in the scene. Therefore, each time the user's pupil position isrecorded looking at a different specific object, the controller 202generates a new data pair between the foreground reference frame 308(e.g., the foreground environment 404) and the external reference frame502 (e.g., the external environment 306). The controller 202 maygenerate the transfer function based on the set of data pairs during theset-up, calibration procedure. The set-up procedure may be repeatedoccasionally to recalibrate the eye tracking system 102.

In another embodiment, the correlation between the foreground reflectionin the lens 106 and the external environment 306 may be determined(e.g., calibrated) based on geometrical calculations. Several propertiesused in the calculation may be constant and determinable, such as thedistance from the pupil 304 to the inner surface 307 of the lens 106,the distance from the imaging device 108 to the eye 302, and the focallength of the imaging device 108. The controller 202 may use geometry tocalculate the correlation (e.g. transfer function) based on theseproperties and a detected angle at which the imaging device 108 captureslight from the pupil 304 reflected off the inner surface 307 of the lens106.

During operation of the eye tracking system 102 the user gazes at thephysical world rather than the stock image of the set-up procedure. Foreach detected position of the pupil 304, the controller 202 may inputthe coordinates of the center point 322 of the pupil 304 (relative tothe foreground reference frame 308) into the transfer function todetermine a location 504 in the external environment 306 to which theeye 302 is gazing. For example, the location 504 may be output by thetransfer function as coordinates in the external reference frame 502.The location 504A in FIG. 5 , indicated by a target symbol that isoverlaid on the image 500, corresponds to the position of the pupil 304in FIGS. 3A and 3B.

FIG. 6 illustrates the same image 500 as FIG. 5 , and identifies asecond location 504B that the user is gazing relative to the image 500.The controller 202 determines the second location 504B by inputting thepositional coordinates of the center point 322 of the pupil 304 shown inFIGS. 4A and 4B into the transfer function. The location 504B in FIG. 6, indicated by the target symbol, is above and to the right of thelocation 504A in FIG. 5 , which correlates with the differences in pupilpositioning between FIGS. 3B and 4B. The locations 504A, 504B may beused to determine the gaze directions 312, 402 and/or an angulardifference between the gaze directions 312, 402.

In an embodiment, the eye tracking system 102 may provide additionalinformation to the user based on the location 504 to which the user isgazing. For example, the eye tracking system 102 may provide informationabout the location 504 to the user as part of an augmented (or mixed)reality platform. For this purpose, the controller 202 may analyze theimage data and identify an object 510 at the location 504 in theexternal environment 306 to which the gaze is directed.

To identify the objects 510 in the external environment 306 associatedwith the gaze locations 504, the controller 202 may use one or moreimage analysis techniques. In an example, the controller 202 usesmachine learning (e.g., artificial intelligence), such as an artificialneural network trained to identify different objects in the externalenvironment. The neural network may be trained to automatically detectand recognize certain specific objects depicted in image data. Theneural network may have multiple layers that represent different groupsor sets of artificial neurons or nodes. An input layer receives an inputimage, such as the image 500 in FIGS. 5 and 6 , or at least portions ofthe image data that includes the locations 504A, 504B. The neurons ornodes are functions performed to identify objects in the input imagedata. The artificial neurons may apply different weights in thefunctions applied to an input image data to attempt to identify orsegment the objects at the locations 504A, 504B.

The neurons in the layers of the neural network may examinecharacteristics of the pixels of the input image data, such as theintensities, colors, or the like, to determine the classificationvectors for the various pixels. The neural network may assign orassociate different pixels with different object classes based on thecharacteristics of the pixels. An object class is a type or category ofan object 510 that may be present in the image 500. For example, a treecan be a first object class, a building can be a different, secondobject class, and a car can be a third object class. A pixel can belabeled (e.g., associated) with probabilities, confidence levels, orscores that the pixel represents various different object classes by avector [a b c], where the values of a, b, and c indicate theprobability, confidence level, or score of the pixel representing eachof different classes of objects or things. The neural network mayexamine the classification vector of each pixel and determine whichobject class has the highest confidence or score for each pixel. Forexample, a first pixel in the input image 500 having a classificationvector of [0.7 0.15 0.15] indicates that the neural network 122calculated a 70% confidence that the first pixel represents a firstobject class, a 15% confidence that the first pixel represents a secondobject class, and a 15% confidence that the first pixel represents athird object class. The neural network may determine that each pixel inthe image 500 represents the object class having the greatest or largestconfidence level or score in the corresponding classification vector forthat pixel. For example, the neural network may determine that the firstpixel described above represents a tree due to the 70% confidence. Thisprocess can be repeated for several, or all, other pixels in the inputimage data.

In the illustrated embodiment, the controller 202 may identify theobject 510A in FIG. 5 as a mountain, and may identify the object 510B inFIG. 6 as the moon. Optionally, the machine learning may be able touniquely identify the objects 510A, 510B, beyond identifying a generaltype or class of the objects 510A, 510B. For example, the controller 202may identify the mountain 510A as a specific mountain, such as PikesPeak.

The controller 202 may perform an operation based on the object 510 thatis identified. For example, the controller 202 may control the displaydevice 208 to display graphic indicia on a display screen visible to theuser wearing the wearable device 100. The graphic indicia relates to theobject 510 that is identified. In an embodiment, the display device 208is mounted on the wearable device 100 (as part of the eye trackingdevice 116), and the display device 208 displays the graphic indiciarelating to the viewed object 510 on at least one of the lenses 106 ofthe wearable device 100.

FIG. 7 illustrates a lens 106 of the wearable device 100 as viewed by auser during operation of the eye tracking system 102 according to anembodiment. FIG. 7 shows a head-up display feature in which the displaydevice 208 display graphic indicia 702 related to the object 510A towhich the user is gazing. For example, the controller 202 may use imageanalysis to identify the object 510A as the mountain named Pikes Peak inColorado, USA. The controller 202 may generate a message that providesinformation about Pikes Peak to the user. The display device 208 isconfigured to display the message as the graphic indicia 702 on the lens106, such that the user can view the graphic indicia 702 while peeringthrough the lens 106. The graphic indicia 702 in the illustrated exampleidentifies the mountain as “Pikes Peak”, and provides the elevation ofthe mountain as 14,115 feet above sea level. The graphic indicia 702 ispresented near the top of the lens 106, to avoid unduly distracting orinterfering with the vision of the user.

In the example shown in FIG. 6 , in which the object 510B is identifiedas the moon, the controller 202 may generate graphic indicia for displayon the lens 106 which provides information about the moon. Theinformation that is presented may include the current phase of the moon,or the like. The user may be able to actuate the input device 206 shownin FIG. 2 to selectively activate and deactivate the head-up, augmenteddisplay feature shown in FIG. 7 .

The controller 202 may perform other operations based on the object 510that is identified. For example, the controller 202 may send anotification to a user computer device based on the identified object510 to which the user is gazing. The notification may provide theidentity of the object 510. In another example, the controller 202 maygenerate an alert. For example, in a driver attentiveness application,the object 510 that is identified may be a smartphone, an infotainmentscreen in a vehicle, or the like. The controller 202 detects that theuser is not being attentive to the route based on the identified object510 and optionally an amount of time that the user's attention isdirected away from the route to the object 510. In response, thecontroller 202 may generate an alert that is provided via vibration ofthe wearable device 100, a sound, a flashing light, or the like. Thealert may prompt the user to direct attention back to the route and theoperation of the vehicle. In another example, the controller 202 maydetect that the driver is inattentive based on a lack of the pupil 304being shown in the reflection 320. For example, if the driver is drowsy,the pupil 304 may be covered by the eyelid. If the controller 202 failsto detect the pupil 304 for at least a threshold time period (e.g., 2seconds, 3 seconds, 5 seconds, etc.), the controller 202 may generatethe alert. In another example operation, the controller 202 may generatea record (e.g., update a log) of the objects 510 that are identified,and store the record in the memory 216. The record may include a seriesof objects 510 that the user looked at over time. The record may be usedfor data analysis, accident reconstruction, and/or the like.

FIG. 8 is a flow chart 800 of a method of eye tracking according to anembodiment. The method may be performed by the eye tracking device 116.For example, the controller 202 may perform at least some of the stepsof the method. The method optionally may include at least one additionalstep than shown, at least one fewer step than shown, and/or at least onedifferent step than shown in FIG. 8 .

At step 802, the controller 202 obtains image data generated by animaging device 108 mounted on a wearable device 100. The wearable device100 includes a lens 106 that is light transmissive and positioned infront of an eye of a user that is wearing the wearable device 100. Theimaging device 108 has a field of view 118 that captures an innersurface 307 of the lens 106 and a reflection 320 of the eye on the innersurface 307. Optionally, the method may include removably attaching theeye tracking system 116, including the imaging device 108, to a frame104 of the wearable device 100.

At step 804, the controller 202 analyzes the image data that isobtained. At step 806, the controller 202 detects a position of a pupilof the eye in the reflection 320 based on the analysis of the imagedata. The controller 202 may detect the position of the pupil in thereflection 320 relative to the eye in the reflection 320, the lens 106in the reflection 320, and/or the field of view 118 of the imagingdevice 108.

At step 808, the controller 202 may determine a correlation between aforeground environment 404 in the image data and an external environment306 in the image data. The correlation may be represented by a transferfunction that is generated by the controller 202. The controller 202 maysegment the image data between the foreground environment 404 and theexternal environment 306 prior to determining the correlation. Theforeground environment 404 includes the inner surface 308 of the lens106 and the reflection 320. The external environment 306 includesobjects disposed beyond the lens 106 relative to the imaging device 108.

At step 810, the controller 202 may determine a location 504 in theexternal environment 306 to which a gaze of the eye is directed based onthe position of the pupil and the correlation between the foregroundenvironment 404 and the external environment 306. At step 812, thecontroller 202 may identify an object 510 at the location 504 in theexternal environment 306 to which the gaze is directed. At step 814, thecontroller 202 may control a display device 208 to display graphicindicia 702 on a display screen visible to the user. The graphic indicia702 may be content that is related to the object 510 that is identified.

The eye tracking system and method described herein provides reliable,accurate eye tracking by detecting a position of the pupil based on areflection of the user's eye on an inner surface of a lens. The hardwareof the eye tracking system may be out of the direct line of sight of theuser, to avoid obstructing the user's vision of the physical world. Theeye tracking system may not emit IR light into the user's eyes. The eyetracking system may be utilized in virtual reality and/or augmentedreality platforms. The eye tracking system may be removably coupled to awearable device to enable retrofitting and/or substitution of thewearable device.

Closing Statements

As will be appreciated by one skilled in the art, various aspects may beembodied as a system, method or computer (device) program product.Accordingly, aspects may take the form of an entirely hardwareembodiment or an embodiment including hardware and software that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects may take the form of a computer (device) programproduct embodied in one or more computer (device) readable storagemedium(s) having computer (device) readable program code embodiedthereon.

Any combination of one or more non-signal computer (device) readablemedium(s) may be utilized. The non-signal medium may be a storagemedium. A storage medium may be, for example, an electronic, magnetic,optical, electromagnetic, infrared, or semiconductor system, apparatus,or device, or any suitable combination of the foregoing. More specificexamples of a storage medium would include the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), a dynamicrandom access memory (DRAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), a portablecompact disc read-only memory (CD-ROM), an optical storage device, amagnetic storage device, or any suitable combination of the foregoing.

Program code for carrying out operations may be written in anycombination of one or more programming languages. The program code mayexecute entirely on a single device, partly on a single device, as astand-alone software package, partly on single device and partly onanother device, or entirely on the other device. In some cases, thedevices may be connected through any type of network, including a localarea network (LAN) or a wide area network (WAN), or the connection maybe made through other devices (for example, through the Internet usingan Internet Service Provider) or through a hard wire connection, such asover a USB connection. For example, a server having a first processor, anetwork interface, and a storage device for storing code may store theprogram code for carrying out the operations and provide this codethrough its network interface via a network to a second device having asecond processor for execution of the code on the second device.

Aspects are described herein with reference to the Figures, whichillustrate example methods, devices and program products according tovarious example embodiments. These program instructions may be providedto a processor of a general purpose computer, special purpose computer,or other programmable data processing device or information handlingdevice to produce a machine, such that the instructions, which executevia a processor of the device implement the functions/acts specified.

The program instructions may also be stored in a device readable mediumthat can direct a device to function in a particular manner, such thatthe instructions stored in the device readable medium produce an articleof manufacture including instructions which implement the function/actspecified. The program instructions may also be loaded onto a device tocause a series of operational steps to be performed on the device toproduce a device implemented process such that the instructions whichexecute on the device provide processes for implementing thefunctions/acts specified.

The units/modules/applications herein may include any processor-based ormicroprocessor-based system including systems using microcontrollers,reduced instruction set computers (RISC), application specificintegrated circuits (ASICs), field-programmable gate arrays (FPGAs),logic circuits, and any other circuit or processor capable of executingthe functions described herein. Additionally, or alternatively, theunits/modules/controllers herein may represent circuit modules that maybe implemented as hardware with associated instructions (for example,software stored on a tangible and non-transitory computer readablestorage medium, such as a computer hard drive, ROM, RAM, or the like)that perform the operations described herein. The above examples areexemplary only, and are thus not intended to limit in any way thedefinition and/or meaning of the term “controller.” Theunits/modules/applications herein may execute a set of instructions thatare stored in one or more storage elements, in order to process data.The storage elements may also store data or other information as desiredor needed. The storage element may be in the form of an informationsource or a physical memory element within the modules/controllersherein. The set of instructions may include various commands thatinstruct the modules/applications herein to perform specific operationssuch as the methods and processes of the various embodiments of thesubject matter described herein. The set of instructions may be in theform of a software program. The software may be in various forms such assystem software or application software. Further, the software may be inthe form of a collection of separate programs or modules, a programmodule within a larger program or a portion of a program module. Thesoftware also may include modular programming in the form ofobject-oriented programming. The processing of input data by theprocessing machine may be in response to user commands, or in responseto results of previous processing, or in response to a request made byanother processing machine.

It is to be understood that the subject matter described herein is notlimited in its application to the details of construction and thearrangement of components set forth in the description herein orillustrated in the drawings hereof. The subject matter described hereinis capable of other embodiments and of being practiced or of beingcarried out in various ways. Also, it is to be understood that thephraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting. The use of“including,” “comprising,” or “having” and variations thereof herein ismeant to encompass the items listed thereafter and equivalents thereofas well as additional items.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the disclosed embodiments (especially in thecontext of the following claims) are to be construed to cover both thesingular and the plural, unless otherwise indicated herein or clearlycontradicted by context. Further, in the following claims, the phrases“at least A or B”, “A and/or B”, and “one or more of A and B” (where “A”and “B” represent claim elements), are used to encompass i) A, ii) B oriii) both A and B.

It is to be understood that the above description is intended to beillustrative, and not restrictive. For example, the above-describedembodiments (and/or aspects thereof) may be used in combination witheach other. In addition, many modifications may be made to adapt aparticular situation or material to the teachings herein withoutdeparting from its scope. While the dimensions, types of materials andcoatings described herein are intended to define various parameters,they are by no means limiting and are illustrative in nature. Many otherembodiments will be apparent to those of skill in the art upon reviewingthe above description. The scope of the embodiments should, therefore,be determined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled. In the appendedclaims, the terms “including” and “in which” are used as theplain-English equivalents of the respective terms “comprising” and“wherein.” Moreover, in the following claims, the terms “first,”“second,” and “third,” etc. are used merely as labels, and are notintended to impose numerical requirements on their objects or order ofexecution on their acts.

What is claimed is:
 1. An eye tracking system comprising: an imagingdevice configured to be mounted on a wearable device, the wearabledevice including at least a first lens that is light transmissive andpositioned in front of at least a first eye of a user that is wearingthe wearable device, the imaging device having a field of view thatcaptures an inner surface of the first lens and a reflection of thefirst eye on the inner surface; a memory configured to store programinstructions; and one or more processors operably connected to thememory and the imaging device, wherein the program instructions areexecutable by the one or more processors to: analyze image datagenerated by the imaging device; and detect a position of a pupil of thefirst eye in the reflection based on the analysis of the image data bycalculating coordinates of a center point of the pupil in the reflectionwithin a foreground reference frame in the image data.
 2. The eyetracking system of claim 1, wherein the imaging device is configured togenerate the image data in a visible wavelength range.
 3. The eyetracking system of claim 1, wherein the imaging device is mounted suchthat the field of view does not directly capture the first eye of theuser.
 4. The eye tracking system of claim 1, further comprising a lightsource configured to be mounted to the wearable device and to emit lighttowards the inner surface of the first lens for providing thereflection.
 5. The eye tracking system of claim 1, wherein the one ormore processors are configured to detect the position of the pupil inthe reflection relative to one or more of the first eye in thereflection, the first lens in the reflection, or the field of view ofthe imaging device.
 6. The eye tracking system of claim 1, wherein theone or more processors are configured to segment the image data to aforeground environment and an external environment, the foregroundenvironment including the inner surface of the first lens and thereflection, the external environment disposed beyond the first lens, theone or more processors configured to determine a location in theexternal environment to which a gaze of the first eye is directed basedon the position of the pupil and a correlation between the foregroundenvironment and the external environment.
 7. The eye tracking system ofclaim 6, wherein the one or more processors are configured to analyze aportion of the image data corresponding to the location in the externalenvironment to which the gaze is directed, and to identify an object atthe location in the external environment based on the portion of theimage data.
 8. The eye tracking system of claim 7, wherein the one ormore processors are configured to perform an operation based on theobject that is identified.
 9. The eye tracking system of claim 8,wherein the one or more processors are configured to control a displaydevice to display graphic indicia on a display screen visible to theuser, the graphic indicia relating to the object that is identified. 10.The eye tracking system of claim 9, further comprising the displaydevice, wherein the display device is mounted on the wearable device anddisplays the graphic indicia relating to the object on at least thefirst lens for viewing by the user.
 11. The eye tracking system of claim1, wherein the one or more processors are configured to determine alocation in an external environment to which a gaze of the first eye isdirected by inputting the coordinates of the center point of the pupilinto a transfer function that represents a correlation between theforeground reference frame and an external reference frame.
 12. The eyetracking system of claim 1, further comprising the wearable device,wherein the wearable device is one of eyeglasses or goggles.
 13. Amethod comprising: analyzing image data generated by an imaging devicemounted on a wearable device, the wearable device including at least afirst lens that is light transmissive and positioned in front of atleast a first eye of a user that is wearing the wearable device, theimaging device having a field of view that captures an inner surface ofthe first lens and a reflection of the first eye on the inner surface;and detecting a position of a pupil of the first eye in the reflectionbased on the analysis of the image data by calculating coordinates of acenter point of the pupil in the reflection within a foregroundreference frame in the image data.
 14. The method of claim 13, furthercomprising removably attaching the imaging device to a frame of thewearable device.
 15. The method of claim 13, wherein detecting theposition of the pupil comprises detecting the position of the pupil inthe reflection relative to one or more of the first eye in thereflection, the first lens in the reflection, or the field of view ofthe imaging device.
 16. The method of claim 13, further comprising:segmenting the image data between a foreground environment and anexternal environment, the foreground environment including the innersurface of the first lens and the reflection, the external environmentdisposed beyond the first lens relative to the imaging device; anddetermining a location in the external environment to which a gaze ofthe first eye is directed based on the position of the pupil and acorrelation between the foreground environment and the externalenvironment.
 17. The method of claim 16, further comprising identifyingan object at the location in the external environment to which the gazeis directed.
 18. The method of claim 17, further comprising controllinga display device to display graphic indicia on a display screen visibleto the user, the graphic indicia relating to the object that isidentified.
 19. A computer program product comprising a non-transitorycomputer readable storage medium, the non-transitory computer readablestorage medium comprising computer executable code configured to beexecuted by one or more processors to: analyze image data generated byan imaging device mounted on a wearable device, the wearable deviceincluding at least a first lens that is light transmissive andpositioned in front of at least a first eye of a user that is wearingthe wearable device, the imaging device having a field of view thatcaptures an inner surface of the first lens and a reflection of thefirst eye on the inner surface; and detect a position of a pupil of thefirst eye in the reflection based on the analysis of the image data bycalculating coordinates of a center point of the pupil in the reflectionwithin a foreground reference frame in the image data.